The Sr. Big Data Java Engineer will develop a data lake streaming platform in Azure, focusing on building scalable and resilient systems. Key responsibilities include:
- Designing and developing Kafka Stream-based Java applications and Spark data pipelines for large-scale data in Azure.
- Writing high-quality Java code, conducting code reviews, and improving processes.
- Troubleshooting performance issues in distributed systems with high data volumes.
- Collaborating with the Product Owner to translate customer requests into development tasks.
- Writing unit tests, deploying code, and ensuring quality and consistency through CI/CD processes.
- Engaging in pair programming and adhering to agile methodologies.
Required skills and experience:
- 7+ years of experience as a Java Developer, with 3+ years in Spark, Kafka, and cloud (AWS or Azure).
- Strong knowledge of distributed systems, Big Data, Spark, Kafka, RESTful APIs, Spring Boot, and cloud technologies.
- Familiarity with testing frameworks (JUnit, Mockito, Cucumber) and CI/CD tools.
- Proficient in agile practices, TDD, and service-oriented architectures.
The ideal candidate is analytical, results-driven, team-oriented, and experienced in designing complex systems for real-time and batch data processing.
